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1.  INTRODUCTION 

State  estimation  is  often  used  in  process  control  and  performance  monitoring 
applications,  where  there  are  many  uncertainties  to  deal  with,  such  as  model 
uncertainties,  measurement  uncertainties  and  various  noise  sources.  In  such  an 
environment,  representing  the  model  state  by  an  (approximated)  probability  density 
function  (pdf)  enables  the  propagation  the  pdf  of  the  system  states  over  time,  often  in 
some  optimal  way.  It  is  most  common  to  use  the  Gaussian  pdf,  as  characterized  by  its 
mean  and  covariance,  to  represent  the  model  state,  process  and  measurement  noises.  It  is 
well  known  that  the  Kalman  Filter  (KF)  propagates  the  mean  and  covariance  of  the  pdf 
of  the  model  state  of  linear  dynamic  systems  in  an  optimal  (minimum  mean  square  error) 
way  [1]. 

In  practice,  real-world  processes  are  rarely  purely  linear.  The  Extended  Kalman 
Filter  (EKF)  is  a  well-known  technique  for  applying  the  KF  to  nonlinear  system.  In  the 
EKF,  the  pdf  is  propagated  through  a  linear  approximation  of  the  system  around  the 
operating  point  at  each  time  instant,  using  the  Jacobian  matrices.  Jacobians  however,  are 
often  difficult  to  calculate,  especially  in  the  case  of  time-sensitive  applications.  In 
addition,  the  EKF’s  linear  system-approximation  often  introduces  state-estimation  errors 
that  diverge  over  time. 

Ulmann  et-al  developed  the  Unscented  Kalman  Filter  (UKF)  [2],  which  operates 
on  non-linear  systems  too.  The  UKF  propagates  the  pdf  in  a  simple  and  effective  way 
and  it  is  accurate  up  to  second  order  in  estimating  mean  and  covariance. 

This  report  describes  the  application  of  the  UKF  combined  with  logic-reasoning 
to  the  monitoring  of  general-purpose  software  using  sparse  probing.  General-purpose 
software  is  typically  not  the  type  of  system  usually  being  monitored  using  KF  techniques 
because:  (i)  it  lacks  well  formed  state-equations  based  on  first-principles,  (ii)  its  state 
space  often  suffers  from  abrupt  changes  and  is  non-monotonic,  and  (iii)  many  general- 
purpose  systems,  such  as  multi-threaded  systems,  are  non-deterministic. 

We  decided  to  use  the  UKF  because  it  incorporates  the  use  of  general-purpose  a-priori 
knowledge.  We  use  logic  reasoning  within  the  a-priori  next  state  UKF  computation. 

The  rest  of  the  paper  is  organized  as  follows.  Section  2  provides  an  overview  of 
KF  and  EKF  techniques.  Section  3  provides  an  overview  of  the  UKF  technique.  Section 
4  describes  the  problem  domain  and  the  suggested  UKF-based  solution  using  the  UKF. 

2.  THE  KALMAN  FILTER  AND  EXTENDED  KALMAN  FILTER 

All  forms  of  the  KF  (i.e.,  KF,  EKF,  UKF,  etc.)  operate  in  two  phases  per 
iteration:  the  prediction  phase  and  correction  phase,  as  depicted  in  Figure  1.  The 
prediction  phase  uses  prediction  formulas,  usually  based  on  first-principles  ( a-priori 
knowledge),  to  predict  the  estimated  state.  An  example  of  a  first-principles  formula  used 
for  this  phase  is  the  representation  of  a  moving  vehicle’s  state  (its  location  and  velocity) 
at  time  t  as  a  function  of  the  state  at  time  t- 1,  using  the  laws  of  motion.  Key  to  the  KF 
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techniques  is  that  one  need  not  record  more  than  a  single  (or  a  limited)  history  of  system 
states. 

In  the  measurement  state,  the  KF  integrates  a-posteriori  knowledge  in  the  form  of 
actual  measurements;  for  the  moving  vehicle  example  a  measurement  is  in  the  form  of  a 
GPS  state  approximation  received  by  the  satellite  triangulation. 

The  components  of  the  KF  estimation  process  of  Figure  1  are: 

•  A  time-update  phase,  based  on  a-priori  information  given  as  linear  update 
equations. 

•  A  measurement-update  phase,  where  measurements  are  incorporated  into 
the  compound  state-estimate;  it  is  based  on  the  Kalman  gain  which 
represents  the  relative  proportion  of  the  measurement  and  time-update 
components  that  are  used  in  the  state-estimate  output 

The  beauty  of  the  KF  set  of  techniques  is  that  it  integrates  both  forms  of 
knowledge  in  an  optimal  way,  resulting  in  a  state  estimate  that  is  superior  to  the  estimate 
generated  by  the  a-priori  or  a-posteriori  infonnation  alone. 


Figure  1.  The  Kalman  Filter 

The  Kalman  Filter  is  designed  for  optimal  estimation  of  linear  systems.  Most 
interesting  real-life  system  estimation  however,  are  concerned  with  non-linear  systems. 
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The  Extended  Kalman  Filter  (EKF)  operates  on  non  linear  but  differentiable  transfer 
functions.  The  EKF  operates  by  approximating  the  state  distribution  as  a  Gaussian 
random  variable  (GRV)  and  then  propagating  it  through  the  first-order  linearization  of 
the  nonlinear  system. 

The  update  equations  for  EKF  are 1 : 

Predict 


Predicted  state  estimate 
Predicted  covariance  estimate 


&k\k-\  ~  f(Xk-\\k-l:Uk-\) 

Pk\k-\  ~  P k-lPk-l\k-lPk-\  +  Qk- 1 


Update 

Innovation  or  measurement  residual 
Innovation  (or  residual)  covariance 
Near-optimal  Kalman  gain 
Updated  state  estimate 
Updated  estimate  covariance 


Vk  =  Zk  -  h(xk\k-i) 

Sk  =  HkPk\k-iHTk  +  Rk 
Kk  =  Pk\k-xHTkS-kl 

*fc|fc  =  +  KkVk 

Pk\k  —  {I  —  KkHk)Pk\k-i 


where  the  state  transition  and  observation  matrices  are  defined  to  be  the  following 
Jacobians: 


F  ~  df 

*  fc-i  —  -5— 

ox 


“fe-llfc-l."*:-! 


rr  _  dfl 


xk\k-l 


Unlike  the  KF,  the  EKF  is  not  generally  optimal  when  applied  to  non-linear 
systems  optimal  estimator.  In  addition,  if  the  original  estimation  is  wrong  then  the 
estimator  often  diverges  quickly.  Also,  calculating  the  Jacobian  is  computationally 
expensive. 

3.  THE  UNSCENTED  KALMAN  FILTER 

While  EKF  approximates  state  distribution  using  Gaussian  Random  Variables 
(GRV)  which  are  propagated  through  a  first-order  linearization  of  the  non-linear  system. 
The  UKF  [2]  replaced  this  technique  with  a  deterministic  sampling  approach,  where  the 
GRV  is  represented  using  a  minimal  set  of  chosen  sample  points.  While  EKF  achieves 
first  order  accuracy  the  UKF  captures  the  posterior  mean  and  covariance  accurately  to 
the  2nd  order  Taylor  series  expansion,  as  demonstrated  by  Jullier  and  Ullman  [2],  The 


1  http://en.wikipedia.org/wiki/Extended_Kalman_filter 
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UKF’s  basic  framework  involves  the  state  estimation  of  a  discrete -time  nonlinear 
dynamic  system, 

xA+i  =  F(xA,  Ufe  VA) 

yA  =  H(xa,  nA) 

where: 

•  xA  represents  the  unobserved  system-state 

•  uA  is  a  known  exogenous  input 

•  yA  is  the  observed  measurement  signal 

•  vA  is  the  process  noise  and  nA  is  the  observation  noise. 

The  block-diagram  of  Figure  2  depicts  such  a  system.  State  estimation  is  concerned  with 
estimating  xA  as  a  Random  Variable  (RV).  It  is  typically  assumed  xA  is  Gaussian;  hence 
state-estimation  is  about  estimating  its  mean  and  covariance. 

The  Unsecented  Transform  (UT)  is  a  method  for  calculating  statistics  of  a  RV  which 
undergors  a  nonlinear  transfonnation  [2].  Suppose  a  RV  x  of  dimension  L  is  propagated 
through  a  nonlinear  function  y  =  f(x),  and  that  x  has  mean  x  and  covariance  Px.  We 
create  a  matrix  X  of  2L+1  sigma  vectors  Xi  as  follows: 

X0=.Y 

Xj  =  .y  +  (vTTTPx)i  for  i=\,...,L 
X  =  x  -  (v'i  +  A  P>:),_L  for  i=L+ 1 , . . .  ,2 L 

Where  A  =  a2  (L+  k)-L  is  a  scaling  parameter,  and  cr  detennines  the  spread  of  the  sigma 
points  around  x .  re  is  also  a  scaling  parameter  usually  set  to  0  or  3 -L. 

These  sigma  points  are  propagated  through  the  non-linear  function  /(),  resulting  in  Yi 
=/(Xj)  The  mean  and  covariance  of  Y  are  then  approximated  using  a  weighted  sample 
mean  and  covariance  of  the  posterior  sigma  points, 

Where  the  weights  are  given  by: 

W0(m)  =  A  /  (L+  A) 

W0(c)  =  A  /  (L+  A)  +  (1-tr2  +  0) 

Wi(m)  =  Wi(c>  =  l/{2  (L+  A) )}  /  =  1,...2 L. 


Figure  3  is  a  block-diagram  illustrating  the  UT  process. 

The  UKF  extends  UT  to  the  recursive  estimation  of  sigma  points  but  where  the  state  RV 
is  redefined  as  the  concatenation  of  the  original  state  RV  and  the  noise  variables: 
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Xka  =  [x/v/n/].  Sigma  points  are  selected  for  the  new  augmented  state  RV,  instead  of 
for  Xk  alone. 


process  noise  measurement  noise 


Figure  2.  Block  diagram  of  a  discrete-time  nonlinear  dynamic  system  [2], 


-V 


/(  ) 


wrtgnca 
sample  mean 


y, 


W endued 
sanple 
covariance 


y=  h  ITT) 


?  t 


{*}=[*  r+rVF.  *-rv'F.] 


Figure  3.  Block  diagram  of  the  UT  [2] 
Java  code  for  the  UKF  is  provided  in  Appendix  B. 
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4.  MONITORING  GENERAL  PURPOSE  SOFTWARE  WITH 
SPARSE  PROBING 

Applying  a  KF  derivative  to  general-purpose  software  is  untraditional  for  the 
following  reasons: 

•  Most  often  is  no  first-principles  fonnula  that  can  be  used  to  compute  the 
a-priori  knowledge.  We  use  reasoning  instead. 

•  The  state  space  often  changes  abruptly  and  non-monotonically. 

•  Many  general-purpose  software  systems  are  not  fully  deterministic,  as  in 
the  case  of  multi-threaded  software;  the  game  example  described  in  the 
sequel  is  such  an  example. 

Nevertheless,  we  show  that  the  UKF  can  be  used  to  monitor  the  state  of  such 
systems,  using  logic  reasoning  for  the  UKF’s  a-priori  knowledge  propagation 
component. 

In  addition  to  our  interest  in  general-purpose  software  systems,  we  are  interested 
in  monitoring  systems  using  probing  that  is  sparse  in  time  and  space.  Temporally  sparse 
probing  is  one  that  occurs  at  a  lower  frequency  than  the  a-priori  knowledge  propagation 
frequency  as  determined  by  the  logic-reasoning  component.  Spatially  sparse  probing 
probes  a  subset  of  the  state  space.  Clearly,  when  probing  general-purpose  systems  whose 
state  space  contains  a  large  volume  of  state  variables,  it  is  not  always  possible  to  probe 
all  such  states. 

4.1.  AN  EXAMPLE:  A  TWO  PLAYER  GAME 

The  example  of  general-purpose  software  used  in  this  report  is  the  following  two-player 
game,  called  the  Shooting  Game.  A  player  is  a  computation  thread  running  concurrently 
to  the  other  with  a  relative  execution  cycle  drift  of  +/-  d%.  To  gain  points,  each  player 
takes  virtual  shots  constrained  by  the  following  rules. 

•  A  virtual  shot  taken  by  player  i  always  counts  towards  that  players  number  of 
shooting  attempts  -  denoted  as  Att(i).  A  virtual  shot  is  sometimes  also  counted  as 
a  hit,  denoted  as  Hit(i). 

•  When  player  i  makes  a  shot  in  counts  as  c(n,  i)  hits,  where  c(n,  i)  =  nl  100  * 
(< abs(Att(j )  -  Hit(i)))  >  1  ?  1:  0),  and  n  is  determined  by  the  stage  of  the  game,  as 
discussed  below.  We  call  this  operation  shoot  and  increment,  although  a  player 
might  not  actually  increment  his  or  her  count,  depending  on  /  Shoot  and 
decrement  is  defined  in  a  dual  manner. 

•  Initially,  both  players  shoot  and  increment  using  n= 50.  The  first  player 
accumulate  two  hits  acquires  the  (unique)  lock. 

•  The  player  that  owns  the  lock  shoots  and  increments  with  n=15.  If  s/he  misses 
s/he  shoots  and  decrements  n= 25. 
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•  The  player  that  does  not  own  the  lock  and  is  waiting  for  the  lock  (i.e.,  s/he  has  2 
hits)  gets  a  5%  chance  to  flip  the  lock,  i.e.,  to  own  the  lock  while  resetting  the 
opponent’s  hit  count  of  (get  the  lock  and  reset  the  opponent)  -  the  chance  is 
based  on  the  reading  of  a  millisecond  timer. 

•  A  player  with  6  or  more  hits,  shoots  with  n=80-(m%2==l  ?  0:  40)  chance  shots, 
where  m  is  time  in  milliseconds. 

•  Once  player  i  reaches  10  hits  s/he  wins  the  game  provided  that  3*Hit(j)>Att(i). 
Otherwise,  the  player  starts  over  the  game,  i.e.,  his  or  hers  Hit(j)  and  Att(i)  values 
are  reset  to  0. 

This  game  also  represents  a  situation  in  which  the  a  general-purpose  software  being 
monitored  is  not  fully  deterministic,  with  the  relative  execution  time  cycle  of  a  player 
drifts  +/-  d%  relatively  to  the  other  player. 

Code  for  the  Player  game  is  available  in  Appendix  A. 

4.2.  UKF  REASONING  COMPONENT  FOR  THE  SHOOTING  GAME 

Our  UKF-based  state  estimation  system  estimates  the  state  of  the  shooting  game 
by  probing,  Hit{  1),  Att(  1 ),  and  Hit( 2).  Note  that  Hit(X)  depends  on  Att(2)  which  is  not 
probed.  This  is  an  example  of  spatially  sparse  probing.  We  also  assume  probing  is 
temporally  sparse. 

We  use  reasoning  within  the//)  component  of  the  UT  transform  of  Figure  3,  the 
component  responsible  for  the  a-priori  transfonnation.  The  reasoning  component  for  the 
game  is  written  in  Java  as  a  collection  of  Propositional  Logic  rules.  The  rules  were 
derived  directly  from  the  game  specification.  Two  examples  of  such  a  rule  are: 

•  If  Player  2  (named  Ted)  has  less  than  two  hits  then  increment  his  count 
according  to  c (50,  2). 

•  If  Ted  has  between  6  and  10  hits  then  increment  his  count  according  to 
c(60,  2).  Note  that  the  game  specification  for  this  range  is  more  elaborate, 
with  a  non  deterministic  choice  between  c(80,  2)  and  c(40,  2);  since  one 
cannot  reason  accurately  about  non-deterministic  events  we  chose  a  fair 
coin  toss  approach,  using  c(60,  2). 

Java  code  for  the  first  reasoning  rule  is: 

if  (Math . floor (nNumberof Hits_TedPS )  <  2)  { 

boolean  b  =  shootTed(50,  nNumberofHits_EdPS ,  nNumberOfAttempts_EdPS , 
nNumberofHits_TedPS ) ; 
if  (b)  { 

nNumberofHits_TedNS++; 

} 

} 

The  Java  code  for  the  reasoning  component  is  provided  in  Appendix  D.  Several 
important  factors  distinguish  the  reasoning  component  from  the  implementation  of  a 
Player: 
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•  Reasoning  is  not  necessarily  complete.  For  example,  our  reasoning  does  not  reason 
about  the  lock.  One  of  many  reasons  for  doing  so  was  that  when  both  players  have 
two  hits  then  non-deterministic  execution  time  determines  which  player  obtains  the 
lock.  Another  example  is  the  absence  of  reasoning  about  non-deterministic  timer 
based  choices  within  the  game,  as  discussed  per  the  second  reasoning  rule  above. 

•  Reasoning  can  be  performed  on  probed  state  variables  (Hit(  1),  Alt(  1 ),  and  Hit( 2)), 
and  un-probed  state  variables  (Att( 2))  alike.  Note  that  as  prescribed  by  the  UKF,  the 
reasoning  component  generates  2L+\  (where  L= 3)  transformations  of  the  state  vector 
per  game  cycle,  while  only  a  single  transform  the  non-probed  variable  is  required  per 
game  cycle. 

4.3.  SHOOTING  STATE  ESTIMATION  RESULTS 

Clearly,  we  cannot  analyze  the  overall  Mean  Square  Error  (MSE)  of  the  proposed 
approach,  because  it  depends  heavily  on  the  nature  of  the  software  system  being 
monitored  (e.g.  the  degree  of  non-determinism  within  the  program)  and  the  quality  of  the 
reasoning  component.  Hence  we  just  show  somewhat  anecdotal  results  for  the  shooting 
game. 

We  played  the  game  with  100  cycles  per  player.  We  perfonned  temporally  sparse 
probing  for  all  3  probed  values  (i.e.,  Hit(  1 ),  Att(  1 0,  and  Hit{ 2)),  where  probing  was 
perfonned  once  per  K=4  cycles  of  the  game  performance.  The  graphs  of  Figure  4 
demonstrate  the  effectiveness  of  the  suggested  approach. 


Y 
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c.  Att{  1) 

Figure  4.  True  (blue)  vs.  UKF  (red)  state  estimates  for  the  Player  game 


Accuracy  results  are  as  follows: 

•  RMSE  of  estimate  for  Hit(l)  is  2.7306244242593904;  when 
normalized  it  is  0.3049048607132327 

•  RMSE  of  measurement  for  Hit(l)  is  3.1352830813181765;  when 
normalized  it  is  0.39191038516477206 

•  RMSE  of  estimate  for  Hit  (2)  is  2.6694017499675726;  when 
normalized  it  is  0.26019228200858496 

•  RMSE  of  measurement  for  Hit (2)  is  3.1064449134018135;  when 
normalized  it  is  0.3883056141752267 

•  RMSE  of  estimate  for  Att(l)  is  2.956916917326075;  when 
normalized  it  is  0.29347430505348177 

•  RMSE  of  measurement  for  Att(l)  is  3.2526911934581184;  when 
normalized  it  is  0.4065863991822648 

Note  the  role  of  the  UKF  a  variable,  which  is  responsible  for  adjusting  the  sigma 

point  distance  from  the  mean. 
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4.4.POST  UKF  REASONING  VS  UKF-REASONING 


Post  UKF  reasoning  is  reasoning  that  is  not  embedded  in  the  UKF  loop.  As 
discussed  earlier,  UKF  reasoning,  which  is  perfonned  within  the  A)  method  of  Figure  3, 
is  performed  on  each  of  the  2Z+1  sigma  points,  i.e.,  points  that  are  approximately  a 
standard  deviation  away  from  the  mean.  In  contrast,  post-UKF  reasoning  is  perfumed 
after  a  UKF  cycle  ends.  Some  examples  of  post-UKF  reasoning  for  the  shooting  game 
are: 


•  If  the  measurement  drops  to  zero  from  a  previous  cycle  then  prefer  the 
reasoning  over  the  UKF  computed  value. 

•  If  UKF  computed  value  is  smaller  than  zero  then  use  zero  instead. 

5.  CONCLUSION 

We  have  demonstrated  an  effective  approach  for  state  estimation  of  general- 
purpose  software  in  which  there  is  no  a-priory  first  principles  state  equation,  the  software 
is  possibly  non-detenninistic,  and  probing  is  sparse  in  space  and  time.  Our  approach  uses 
the  UKF  augmented  with  propositional  logic  reasoning  with  a  domain  of  discourse  that 
contains  both  the  measured  (probed)  state  variables  within  the  UKF  and  the  unmeasured 
ones. 

Given  the  nature  of  the  sponsored  research,  we  were  unable  to  specific 
assumptions  about  the  nature  of  the  system  under-estimation.  We  hope  to  provide  more 
specific  technique  when  we  focus  on  more  specific  types  of  software  systems. 

6.  CONTINUING  RESEARCH 

We  plan  on  developing  and  demonstrating  a  technique  for  monitoring  time  series 
properties  pertaining-to  or  asserting-about)  the  statistics  of  a  software  system  being 
estimated  via  the  above  mentioned  technique.  Since  the  underlying  system  states  are 
actually  random  variables,  one  can  only  reason  about  them  probabilistically.  Our  planned 
technique  will  allow  the  specification  of  properties  using  existing  formal  specification 
techniques  and  methodologies,  yet  using  a  monitoring  technique  that  caters  for  random 
variables  rather  than  detenninistic  propositions. 

For  example,  consider  the  shooting-game  property  PI:  “ Hit(l )  +  Hit(2)  <=  2  for 
no  more  than  3  consecutive  cyc/es,\  To  monitor  this  temporal  property  we  can  use  he 
following,  existing  techniques: 

1.  Represent  the  property  as  a  fonnal  specification  using  a  specification  language 
such  as  Statechart  assertions,  and  then  monitor  the  measured  values  using  a 
corresponding  runtime  verification  tool,  such  as  the  StateRover  [4,  5], 

2.  Represent  and  monitor  the  property  as  in  (1)  but  monitor  the  mean  value  of  the 
estimated  (hidden)  values  rather  than  measured  values. 

3.  Represent  the  property  as  in  (1  and  2)  but  monitor  the  mean  and  covariance 
values.  Monitoring  in  this  case  is  not  deterministic,  which  will  require  a  new 
approach,  to  be  developed. 
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Clearly,  approaches  (2)  and  (3)  monitor  values  that  are  more  accurate  than  the  values 
monitored  by  approach  (1),  resulting  in  more  accurate  monitoring.  Approach  (3)  is 
more  powerful  than  (2)  in  the  following  sense.  Suppose  the  mean  values  of  Hit(l) 
and  Hit(2)  conform  to  PI  but  other  values,  which  are  rather  probable  (e.g.  probability 
of  0.7)  do  not  confonn  to  PI.  In  such  a  case,  the  end  user  observing  the  monitored 
results  might  be  interested  to  know  that  such  a  possibility  exists  (the  possibility  and 
its  likelihood). 
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8.  APPENDIX  A.  THE  GAME 

package  game; 

import  j ava . util . Calendar ; 
import  j ava . util . Random; 
import  stateestimation . GameObserver ; 
I  *  * 

* 

*  @author  dorondru 
*/ 


public  class  Player  implements  Runnable  { 

public  final  static  int  PLAYERS_INVOCATION_SKEW  =5;  //  from  -2  to  2 
public  static  final  double  RATIO  =  3; 

Random  random; 

Random  randomMillisForDebug; 

Random  rSleepSkew; 
private  boolean  bFinished; 

int  nNoOfCycles; 
boolean  bPauseForDebug; 
public  static  final  int  ED  =  1; 
public  static  final  int  TED  =  2; 


private 

private 

private 

private 

private 

private 

private 


final  int  nID; 

Player  friend; 

MyLock  sharedLock; 

int  nNumberOf Hits  =  0; 

int  nNumberOf Attempts  =  0; 

int  nNumberOfWins  =  0; 

boolean  singleCycle2getMillisec 


false; 


public  Player (int  nID,  Random  rSleepSkew,  int  nNoOfCycles,  MyLock 
sharedLock)  { 

this.nID  =  nID; 

random  =  new  Random ( 100); 

randomMillisForDebug  =  new  Random(150); 

this . rSleepSkew  =  rSleepSkew; 

this . nNoOfCycles  =  nNoOfCycles; 

this . sharedLock  =  sharedLock; 

bFinished  =  false; 

resetMe (0, "construction"  ,  true)  ; 

nNoOfCycles  =  -1;  //  temporary 

this . bPauseForDebug  =  false; 

} 


public  void  setFriend ( Player  friend)  { 
this. friend  =  friend; 

} 


public  int  getID()  { 
return  this.nID; 

} 


public  MyLock  getSharedLock ( )  { 

return  sharedLock; 

} 


public  int  getNumberOf Hits ( )  { 

return  nNumberOf Hits ; 

} 
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public  int  getNumberOf Attempts ( )  { 

return  nNumberOf Attempts ; 

} 


public  int  getNumberOfWins ( )  { 

return  nNumberOfWins ; 

} 


public  boolean  getSingleCycle2getMillisec ( )  { 

return  singleCycle2getMillisec; 

} 


@Override 

public  void  run ( )  { 

for  (int  i  =  0;  i  <  nNoOfCycles;  i++)  { 

try  { 

while  (bPauseForDebug)  { 

Thread. sleep (10) ; 

} 

int  nSleepSkew  =  0; 
if  (rSleepSkew  !=  null)  { 

nSleepSkew  =  rSleepSkew . nextlnt ( PLAYERS_INVOCATION_SKEW) 
-  PLAYERS_INVOCATION_SKEW/2;  // [ -2 , 2 ] 

} 

Thread. sleep ( 10+  nSleepSkew); 

}  catch  (InterruptedException  e)  {} 
singleCycle  (i) ; 


} 

bFinished  =  true;  //  for  the  case  I  won 


boolean  isFinishedO  { 
return  bFinished; 

} 


//  implements  single  cycle  in  game 
public  void  singleCycle ( int  nCycle)  { 

try  { 

printDebug (nCycle,  " - nNumberOf Hits="  +  nNumberOfHits  +  "; 

nNumberOfAttempts="  +  nNumberOfAttempts  +  ";  SingleCycle  #"  +  nCycle  +  nID=" 
+  nID)  ; 

singleCycle2getMillisec  =  false; 
if  (nNumberOfHits  <  2)  { 

boolean  b  =  shoot(50,  nCycle);  //  50%  chance 
if  (b)  { 

nNumberOf Hit s++; 

printDebug2 (nCycle, "incrementing  #1:  nNumberOf Hits="  + 
nNumberOfHits  +  nID="  +  nID) ; 

} 

} 

else  if  (nNumberOfHits  >=  2  &&  nNumberOfHits  <  6)  { 

boolean  locklsMine  =  false; 
if  (nNumberOfHits  ==  2 )  { 

sharedLock . tryLock (nID) ; 

printDebug2 (nCycle, "tring  to  get  lock  -  nID="  +  nID) ; 

} 

locklsMine  =  sharedLock. isLockOwnedByThisPlayer (nID) ; 
if  (locklsMine)  { 
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printDebug2 (nCycle, "LocklsMine  -  nID="  +  nID) ; 
boolean  b  =  shoot  (75,  nCycle); 
if  (b)  { 

nNumberOf Hits++  ; 

printDebug2 (nCycle, "incrementing  #2 : 
nNumberOfHits="  +  nNumberOfHits  +  nID="  +  nID) ; 

}  else  { 

b  =  shoot (25,  nCycle); 

if  (b  &&  nNumberOfHits>0)  nNumberOfHits--; 

} 

}  else  {111  don't  have  the  lock 

printDebug2 (nCycle, "Lock  is  NOT  Mine  -  nID="  +  nID) ; 
boolean  b  =  false; 

if  ( !b)  {  //  I  missed  -  check  time  in  millis 

long  millis  =  getMillisec (nCycle) ; 
if  (millis  <  0)  millis  =  0-millis;//  unexpected 
singleCycle2getMillisec  =  true; 
int  n5  =  (int) (millis%100) ; 
if  (n5  <  5)  { 

sharedLock. flipLock (nID) ; 

this . friend. resetMe (nCycle,  "Reset  due  to  5% 

chance  lock  flip",  false); 

printDebug2 (nCycle, "Incrementing  my  friends 

count  -  I'm  nID="  +  nID)  ; 


}  //  (nNumberOfHits  >=  2  &&  nNumberOfHits  <  6) 
else  if  (nNumberOfHits  >=  6  &&  nNumberOfHits  <=  10)  {//  here 

nNumberOfHits  =  6  or  0,  but  0  will  be  treated  in  the  next  cycle 

printDebug2 (nCycle, "nNumberOfHits  between  6  and  10="  + 
nNumberOfHits  +  "  -  nID="  +  nID)  ; 

if  (nNumberOfHits  ==  6)  { 

printDebug2 (nCycle, "unlocking  lock  (if  locked)  -  nID="  + 

nID)  ; 

unlock ( ) ; 


singleCycle2getMillisec  =  true; 
long  millis  =  getMillisec (nCycle) ; 
int  m  =  millis%2==l ?0 : 40 ; 

boolean  b  =  shoot(80-m,  nCycle);  //  80-m%  chance 
if  (b)  { 

printDebug2 (nCycle, "incrementing;  nNumberOfHits="  + 
nNumberOfHits  +  ";-  nID="  +  nID) ; 

nNumberOf Hits++ ; 
if  (nNumberOfHits  ==  10)  { 

int  scaledNoOfHits  = 

(int) ( (double) nNumberOfHits  *  RATIO); 

if  (nNumberOf Attempts  <  scaledNoOfHits)  {  // 

start  over 

System. out . println ( "reset  due  to 
nNumberOfAttempts  >  scaledNoOfHits  for  nID="  +  nID) ; 

resetMe (nCycle,  "reset  due  to 

nNumberOfAttempts  >  scaledNoOfHits  for  nID="  +  nID,  true) ; 

} 

else  { 

//winner:  when  not  learning  mode  then 
finish,  otherwise:  just  do  another  round  of  learning 

nNumberOfWins++ ; 
resetToGameStart (nCycle) ; 
printDebug2 (nCycle,  "nID="+nID+  "  has 
nNumberOfWins="+nNumberOfWins  +  "  wins"); 
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} 

}  catch  (Exception  e)  { 

System . err . println ( "Exception  in  Player . singleCycle (nCycle) ,  for 
nCycle="  +  nCycle  +  +  e) ; 

} 

} 


//  f  (n,  i)  =  n/100  *  (abs ( #attempts-of-player- j  minus  #-hits-of-player-i) ) 
>1?  true:false 

boolean  shoot (int  nProbability,  int  nCycle)  { 
nNumberOf Attempt s++; 

int  nNoAttemptsOtherPlayer  =  friend . getNumberOf Attempts () ; 
int  nDiff  =  Math . abs (nNoAttemptsOtherPlayer  -  nNumberOf Hits ) ; 
double  d  =  (double) nDiff  *  (double) nProbability  /  100. Of; 
if  (d  >  1.0)  return  true; 
return  false; 


synchronized  void  unlock ()  { 

sharedLock . unlock ( ) ; 

} 


void  resetToGameStart ( int  nCycle)  { 

resetMe (nCycle, "reseting  due  to  game  start  -  nID="  +  nID,  true); 
friend . resetMe (nCycle, "reseting  due  to  game  start  -  nID="  +  nID,  true) ; 

} 


void  resetMe (int  nCycle,  String  sMsg,  boolean  bResetNumberOfAttempts)  { 
try  { 

printDebug (nCycle, " - >reseting  due  to  "  +  sMsg  +  "  -  nID="  + 

nID)  ; 

nNumberOfHits  =  0; 

if  (bResetNumberOfAttempts)  nNumberOf Attempts  =  0; 
if  ( sharedLock . isLockOwnedByThisPlayer (nID)  )  unlock (); 

}  catch  (Exception  e)  { 

System . err . println ( "Exception  in  resetMe;  "  +  e) ; 


private  long  getMillisec (int  nCycle)  { 

Calendar  lCDateTime  =  Calendar . getlnstance () ; 
long  1  =  lCDateTime . getTimelnMillis () ; 

if  (GameObserver . LOCK_STEP)  1  =  randomMillisForDebug . nextlnt ( 100) ; // 
capturing  random  behavior  for  deterministic  debug 
return  1; 


void  setGameOver ( )  { 

bFinished  =  true; 

} 


void  setPauseForDebug ( )  { 

bPauseForDebug  =  true; 
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void  resetPauseForDebug ( )  { 

bPauseForDebug  =  false; 

} 

public  static  void  printDebug ( int  i.  String  s)  { 
System. out . println ( s) ; 

} 

public  static  void  printDebug2 (int  i,  String  s)  { 
//printDebug ( i,  s) ; 

} 

} 
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9.  APPENDIX  B  -  UKF  IN  JAVA2 


package  stateestimation; 

import  static  j ava . lang . Math . * ; 
public  class  UnscentedKalmanFilter 
{ 

//Instance  variables 
/**  Number  of  states.  */ 

private  int  L; 

/**  Number  of  measurements*/ 

private  int  m; 

/**  Tunable.  */ 

private  double  alpha; 

/**  Tunable.  */ 

private  double  ki; 

/**  Tunable.  */ 

private  double  beta; 

/**  Scaling  factor.  */ 

private  double  lambda; 

/**  Scaling  factor.  */ 

private  double  c; 

/**  Weights  for  means.  */ 
private  Matrix  Wm; 

/**  Weights  for  covariance .  */ 

private  Matrix  Wc; 

//Extensive  generall  debug:  0  =  no  debug,  1  =  debug  on  PC,  2  =  debug  on 

brick 

private  static  final  int  DEBUG  =  0; 
private  final  boolean  DEBUG  LIGHT  =  false; 
private  final  boolean  DEBUG2  =  false;  //ukf.utO 
private  final  boolean  DEBUG3  =  false;  //ukf. sigmas () */ 

/*  Constructor 

*  @param  L  number  of  states,  Doron :  N  in  paper=_Paper 

*  @param  m  number  of  measurements 
*/ 

public  UnscentedKalmanFilter (int  L,  int  m) 

{ 

//Logger . println ( "Creating  UKF  for  tracking"); 

this.L  =  L; 
this.m  =  m; 

alpha=lf;  //  alpha  should  change  according  to  the  target 

software  system 

ki=0;  //default 

beta=2f; //pow (alpha,  2)  -0.9f;  //lower  bound  -2;  10  5  10000  - 
2*  -1  0  1  de f : 2 ;  default,  tunable 

lambda=pow(alpha,  2)*(L+ki)-L; 
c=L+lambda; 

2http://code. google. com/p/cats/source/browse/trunk/simulation/particle_filter/GSim/src/GSim/Unscent 
edKalmanF  i  I  ter.j  ava?t — 65 1 
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Wm  =  new  Matrix(l,  (2*L+1),  0.5/c); 

Wm. set (0, 0 , lambda/ c) ; 

Wc=Wm. copy ( ) ; 

Wc.set(0,0,  Wc.get(0,0)  +  1  -  pow(alpha,  2)  +  beta) ; 
c=sgrt(c);  } 


/  -k  -k 


' k 
k 
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UKF,  Unscented  Kalman  Filter,  for  nonlinear  dynamic  systems. 

[x,  P]  =  ukf (f,x,P,h,z,Q,R)  returns  state  estimate  x  and  state 
P 

for  nonlinear  dynamic  system  (for  simplicity,  noises  are  assumed  as 


where  w  ~ 
v  ~ 

@param  f 
@param  x 
@param  P 
Uparam  h 
Sparam  z 
@param  Q 
@param  R 
@ return 


x_k+l  =  f (x_k)  +  w_k 
z_k  =  h(x_k)  +  v_k 

N(0,Q)  meaning  w  is  gaussian  noise  with  covariance  Q  and 

N(0,R)  meaning  v  is  gaussian  noise  with  covariance  R. 

function  handle  for  f (x) ,  nonlinear  state  equations 

"a  priori”  state  estimate 

"a  priori"  estimated  state  covariance 

fanction  handle  for  h(x),  measurement  equation 

current  measurement 

process  noise  covariance 

measurement  noise  covariance 

"a  posteriori"  state  estimate  and  P:  "a  posteriori"  state 


covariance 

*/ 


public  Matrix []  run_ukf ( IFunction  f.  Matrix []  x_and_P,  IFunction  h. 
Matrix  z.  Matrix  Q,  Matrix  R)  throws  Exception  //Cholesky  can  throw  exception 
{ 


//long  time_start_run_ukf  =  System. currentTimeMillis () ; 
if  ( DEBUG  ! =  0) 

{ _ debug ( "Entering  ukf  with  the  following  parameters:"); 


debug ( "Debug :  ukf,  x  dim=  "  + 


X 

and  P[0] . getRowDimension ( )  +  "  x  "  +  x  and  P[0] . getColumnDimension ( )  +  ", 

x= 

n 

-L 

debug (Matlab . MatrixToString (  x  and  P[0])  ); 

debug ( "Debug :  ukf,  P  dim=  "  + 

X 

and  P [ 1 ] . getRowDimension ( )  +  "  x  "  +  x  and  P[l] . getColumnDimension ( )  +  ", 

p= 

n 

-L 

debug (Matlab . MatrixToString (x  and  P[l])); 

debug ( "Debug :  ukf,  z  dim=  "  +  z . getRowDimension ( )  + 

II 

X 

ii 

+ 

z . getColumnDimension ( )  +  ",  z=  "); 

debug (Matlab . MatrixToString (z)  )  ; 

debug ( "Debug :  ukf,  Q  dim=  "  +  Q . getRowDimension ( )  + 

II 

X 

ii 

+ 

Q . getColumnDimension ( )  +  ",  Q=  "); 

debug (Matlab . MatrixToString (Q) ) ; 

debug ( "Debug :  ukf,  R  dim=  "  +  R. getRowDimension ()  + 

II 

X 

ii 

+ 

R. getColumnDimension ( )  +  ",  R=  "); 

debug (Matlab . MatrixToString (R) ) ; 

debug ( "Debug :  ukf,  Wm  dim=  "  +  Wm . getRowDimension ( ) 

+ 

II 

X 

II 

+  Wm . getColumnDimension ( )  +  ",  Wm=  "); 

debug  (Matlab .  MatrixToString  (Vim)  )  ; 

debug ( "Debug :  ukf,  Wc  dim=  "  +  Wc . getRowDimension ( ) 

+ 

II 

X 

II 

+  Wc . getColumnDimension ( )  +  ",  Wc=  "); 

_ debug  (Matlab .  MatrixToString  (Vic)  )  ; 

_ debug( "Starting  calculations") ; 

_ if  (x  and  P [ 0 ]  . getRowDimension ( )  !=  L  |  1 

x  and  P [ 0 ] ■ getColumnDimension ( )  !=  1)  debug ( "WARNING :  The  dimension  of  the 
state  vector  (matrix)  x  is  incorrect!  Expected  dim  =  "  +  L  +"  x  1"  ); 
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_ if_  (x  and  P [ 1 ] , getRowDimension ( )  !=  L  | | 

x  and  P [ 1 ] . getColumnDimension ( )  !=  L)  debug ( "WARNING :  The  dimension  of  the 


state 

covariance  matrix  P  is  incorrect!  Expected 

dim  =  "  +  L  +"  x  "  +  L); 

if  (Q. getRowDimension ( )  ! 

!=  L  ||  Q . getColumnDimension ( ) 

!=  L) 

debug ("WARNING 

:  The  dimension  of  the  covariance  of  process  matrix  Q  is 

incorrect!  Expected 

dim  =  "  +  L  +"  x  "  +  L)  ; 

if  ( z . getRowDimension ( )  ! 

1=  m  ||  z . getColumnDimension ( ) 

!=  1) 

debug ("WARNING 

:  The  dimension  of  the  measurement  vector  (matrix)  z  is 

incorrect!  Expected 

dim  =  "  +  m  +"  x  1"); 

if  (R. getRowDimension ( )  ! 

1=  m  ||  R . getColumnDimension ( ) 

!  =  m) 

debug ("WARNING 

:  The  dimension  of  the  covariance  of  measurement  matrix  P 

is  incorrect!  Expected  dim  =  "  +  m  +"  x  "  +  m) ; 
} 


Matrix  X  =  sigmas (x_and_P [0 ], x_and_P [1] , c) ;  //sigma  points 
around  x,  NB:  c  has  been  set  in  the  constructor 

if  (DEBUG  ! =  0) 

I 

_ debug ( "Debug :  ukf,  X  dim=  "  +  X , getRowDimension ( )  +  "  x 

"  +  X . getColumnDimension ( )  +  ",  X  (after  sigmas ()  )=  "); 

_ debug (Matlab . MatrixToString (X) ) ; 

} 


Matrix []  ut_f_matrices=  ut ( f , X, Wm, Wc, L, Q) ;  / /unscented 

transformation  of  process 

Matrix  xl  =  ut_f_matrices [ 0 ] ; 

Matrix  XI  =  ut_f_matrices [ 1 ] ; 

Matrix  PI  =  ut_f_matrices [2 ] ; 

Matrix  X2  =  ut_f_matrices [3 ] ; 

if  ( DEBUG  ! =  0) 

I 

debug ( "Debug :  ukf,  xl  dim=  "  +  xl . getRowDimension ( )  +  " 


x  "  +  xl , getColumnDimension ( )  +  ",  xl=  "); 

debug (Matlab . MatrixToString (xl) ) ; 


debug ( "Debug :  ukf,  Xl  dim=  "  +  Xl . getRowDimension ( ) 

+  " 

x  "  + 

Xl . getColumnDimension ( )  +  ",  Xl=  "); 

debug (Matlab . MatrixToString (Xl ) ) ; 

debug ( "Debug :  ukf,  PI  dim=  "  +  PI . getRowDimension ( ) 

+  " 

x  "  + 

PI . getColumnDimension ( )  +  ",  Pl=  "); 

debug (Matlab . MatrixToString (PI ) ) ; 

debug ( "Debug :  ukf,  X2  dim=  "  +  X2 . getRowDimension ( ) 

+  " 

X  "  + 

X2 . getColumnDimension ( )  +  ",  X2=  "); 

debug (Matlab . MatrixToString (X2)  )  ; 

I 


Matrix []  ut_h_matrices  =ut (h, Xl , Wm, Wc, m,  R) ;  / /unscented 

transformation  of  measurments 

Matrix  zl  =  ut_h_matrices [ 0 ] ; 

Matrix  Zl  =  ut_h_matrices [ 1 ] ; 

Matrix  P2  =  ut_h_matrices [2 ] ; 

Matrix  Z2  =  ut_h_matrices [3 ] ; 

//long  time_af ter_ut_h  =  System. currentTimeMillis () ; 
if  (DEBUG  ! =  0) 

I 

_ debug ( "Debug :  ukf,  zl  dim=  "  +  zl . getRowDimension ( )  +  " 

x  "  +  zl . getColumnDimension ( )  +  ",  zl=  "); 

debug (Matlab . MatrixToString (zl)  )  ; 
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+  Z1 ■ getRowDimension ( )  + 


debug ( "Debug :  ukf,  Zl  dim=  " 


x  "  +  Zl . getColumnDimension ( )  +  ",  Zl=  "); 

debug (Matlab . MatrixToString ( Zl ) 

) ; 

debug ("Debug:  ukf. 

P2  dim=  "  + 

P2 . getRowDimension ( )  +  " 

x  "  +  P2 . getColumnDimension ( )  +  ",  P2=  "); 

debug (Matlab . MatrixToString ( P2) 

) ; 

debug ("Debug:  ukf, 

Z2  dim=  "  + 

Z2 . getRowDimension ( )  +  " 

x  "  +  Z2 . getColumnDimension ( )  +  ",  Z2=  "); 

debug (Matlab . Ma trixToString ( Z2 )  )  ; 


I 


Matrix  P12  =  (  X2 . times (Matlab . diagFromColumr (Wc) ) 

). times ( Z2 . transpose ()) ;  //transformed  cross-covariance 
if  ( DEBUG  ! =  0) 

I 

_ debug ( "Debug :  ukf,  P12  dim-  "  +  P12 ■ getRowDimension ( )  + 

"  x  "  +  P12 , getColumnDimension ( )  +  ",  P12=  "); 

_ debug (Matlab . MatrixToString (T12) ) ; 

} 

Matrix  K  =  Matrix . identi ty (P12 . getRowDimension ()  ,  P12 . getColumnDimension ()) ; 
try  {  / /  Doron  -  added 

K  =  P2 . transpose (). solve (P12 . transpose ()). transpose  () ; 

}  catch  (Exception  e)  { 

//  Doron  -  added  —  just  keep  diag  K 

} 


if  ( DEBUG  ! =  0) 

I 

_ debug ( "Debug :  ukf,  K  dim=  "  +  K . getRowDimension ( )  +  "  x 

"  +  K. getColumnDimension ( )  +  ",  K=  "); 

_ debug (Matlab ■ MatrixToString (K) ) ; 

} 


x_and_P[0]  =  xl.plus(  K. times ( z .minus ( zl ) )  );  //state  update, 
x_and_P[l]  =  Pl.minus(  K. times (P12 . transpose () )  );  / / covariance 

update, 

if  ( DEBUG  ! =  0) 


I 

debug ( "Leaving  ukf  with  the  following  results:"); 


debug ("Debug:  ukf,  x  updated  dim=  "  + 

x  and  P[0] 

. getRowDimension ( )  +  "  x  "  +  x  and  P[0] . getColumnDimension ( )  +  ", 

x  updated= 

debug (Matlab .  MatrixToString (x  and  P[0])); 

debug ("Debug:  ukf,  P  updated  dim=  "  + 

x  and  P [ 1 ] 

. getRowDimension ( )  +  "  x  "  +  x  and  P [ 1 ] . getColumnDimension ( )  +  ", 

P  updated^  " ) ; 

_ debug (Matlab ■ MatrixToString (x  and  P [ 1 ] ) ) ; 

} 

return  xandP; 

}//end  of  ukf ( ) 


/  -k  -k 

*  Unscented  Transformation 

*  @param  f  nonlinear  map 

*  Qparam  X  sigma  points 

*  @param  Wm  weights  for  mean 

*  Uparam  Wc  weights  for  covariance 

*  @param  n  number  of  outputs  of  f 

*  Sparam  R  additive  covariance 

*  Sreturn  y:  transformed  mean,  Y:  transformed  sampling  points,  P: 
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transformed  covariance,  Y1 :  transformed  deviations 
*/ 

private  Matrix []  ut(IFunction  func.  Matrix  X,  Matrix  Wm,  Matrix  Wc,  int 
n.  Matrix  R)  throws  Exception 
{ 

int  L  =  X . getColumnDimension () ; 

Matrix  y  =  Matlab . zeros (n , 1 ) ; 

Matrix  Y  =  Matlab . zeros (n, L) ; 

int  X_row_dim  =  X . getRowDimension (); points 

int  Y_row_dim  =  Y . getRowDimension () ; 
int  y_row_dim  =  y. getRowDimension () ; 

Matrix  row_in_X; 
for  (int  k=0;  k<L;  k 


//for  all  columns  in  X,  compute  f state  for  the  given 
row  vector  and  put  the  result  in  Y 

row_in_X  =  X . getMatrix ( 0,  X_row_dim-l ,  k,  k); 

Y . setMatrix ( 0,  Y_row_dim-l,  k,  k,  func . eval (row_in_X) 

)  ; 


y.setMatrix(  0,  y_row_dim-l,  0,  0,  (  ( Y . getMatrix ( 0 , 

Y_row_dim-l,  k,  k) ). times (Wm . get ( 0 ,  k) )  ).plus(y)  ); 


} 


Matrix  Y1  =  Y. minus (  y. times ( 

Matlab . ones ( 1 , Y . getColumnDimension () )  )  ); 

Matrix  P  =  Y1 . times (Matlab . diagFromColumn (Wc) )  ; 

P  =  P. times (Y1 . transpose ()) ; 

P.plusEquals (R) ; 

//create  output  matrix  array 
Matrix []  output  =  {y,Y,P,Yl}; 
return  output; 

}//end  of  ut () 

I  -k  -k 
kc 

*  Sigma  points  around  reference  point 

*  Sparam  x  reference  point 

*  @param  P  covariance 

*  @param  c  coefficient 

*  dreturn  Sigma  points  //  Doron:  there  s.b.  2L+1  (2N+1,  using 

paper=  Paper  terminology)  points 

*  / 

private  Matrix  sigmas (Matrix  x,  Matrix  P,  double  c)  throws  Exception 

//Cholesky  can  throw  exception 

{ 

Matrix  A  =  new  Matrix (  Cholesky . cholesky (  P.getArrayO  )  );. 

A  =  A. times (c) ; 

A. transpose  ( ) ; 

int  n  =  x . getRowDimension  (); 


//Create  Y 

Matrix  Y  =  new  Matrix(n,  n,  1); 
for  (int  j=0;  j<n;  j++)  //columns 
{ 

Y. setMatrix (0,  n-1,  j,  j,  x) ; 

} 
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//Create  X 

Matrix  X  =  new  Matrix(n, (1+n+n)); 

X. setMatrix (0,  n-1,  0,  0,  x) ; 

Matrix  Y_plus_A  =  Y. plus (A); 

X . setMatrix ( 0,  n-1,  1,  n,  Y_plus_A) ; 

Matrix  Y_minus_A  =  Y. minus (A); 

X . setMatrix ( 0,  n-1,  n+1,  n+n,  Y_minus_A 

return  X; 

}//end  of  sigmas () 


/  -k  -k 

*  Prints  a  filter  object 
*/ 

public  String  toStringO 

{ 


String  s 

s  =  "L="+L  +  "\n  m  =  "  +  m  +  "\n"; 

s  +=  "  alpha  =  "  +  alpha  +  "\n  ki  =  "  +  ki  +  "\n  beta  = 

beta  + 


"\n  lambda  =  "  +  lambda  +  "\n  c  =  "  +  c  +  "\n  Wm  =  "  + 
Matlab  .  MatrixToString  (Wm)  +  "  Wc  =  "  +  Matlab .  Ma  trixToString  (Vie)  ; 


"  + 


} 


return  s ; 


private  static  void  debug (String  s) 

{ 

if  ( DEBUG  -  0) 

return; 

if ( DEBUG  ==  1)  { 

_ System. out . print In ( s ) ; 

_ L 

_ else 

_ Logger , prin tin ( s ) ; 

} 

}//end  of  class 


22 


10.  APPENDIX  C  -  REASONING  COMPONENT 


The  reasoning  component  for  the  game  is  written  in  Java  as  a  collection  of 
Propositional  Logic  rules.  The  rules  were  derived  directly  from  the  game  specification. 

public  Matrix  eval (Matrix  x)  { 

Matrix  output  =  new 

Matrix (x . getRowDimension ( ) , x . getColumnDimension ( ) ) ; 

double  nNumberofHits_EdPS  =  x.get(0,  0)  ; 

double  nNumberofHits_EdNS  =  nNumberof Hits_EdPS ; 

double  nNumberOf Attempts_EdPS  =  x.get(l,  0)  ; 

double  nNumberof Attempts_EdNS  =  nNumberof Attempts_EdPS+l ; 

double  nNumberofHits_TedPS  =  x.get(2,  0); 
double  nNumberof Hits_TedNS  =  nNumberofHits_TedPS ; 

if  (Math . floor (nNumberof Hits_EdPS )  >  2  && 

nNumberofHits_EdPS  <  6  &&  Math. floor (nNumberofHits_TedPS)  >  2  && 
nNumberofHits_TedPS  <  6)  { 

if  (nNumberofHits_TedPS  <  nNumberof Hits_EdPS )  { 

nNumberof Hits_TedPS  =  2.0; 

}  else  { 

nNumberof Hits  EdPS  =  2.0; 


//  Ted 

if  (Math . floor (nNumberofHits_TedPS)  <  2)  { 

boolean  b  =  shootTed(50,  nNumberofHits^EdPS , 
nNumberOfAttempts_EdPS ,  nNumberofHits_TedPS ) ; 

if  (b)  { 

nNumberof Hits_TedNS++  ; 

} 

} 

else  if  (nNumberofHits_TedPS  <  6.0)  { 

//boolean  locklsMine  =  false;  //  not  modeling  lock 
if  (true  /*lockIsMine*/ )  { 

boolean  b  =  shootTed(75, 

nNumberofHits_EdPS ,  nNumberOfAttempts_EdPS,  nNumberof Hits_TedPS ) ; 

if  (b)  { 

nNumberof Hits_TedNS++ ; 

}  else  { 


nNumberof Hi ts_EdPS ,  nNumberof At tempts_EdPS, 
nNumberofHits  TedNS--; 


b  =  shootTed(25, 
nNumberof Hits_TedPS )  ; 

if  (b  &&  nNumberofHits  TedNS  >  0) 


} 

}  else  (  //  I  don't  have  the  lock 

//  not  implemented  in  Estimator 


_ L 

} 

else  if  (nNumberofHits_TedPS  >=  6  && 

Math . ceil (nNumberofHits_TedPS)  <  10)  { 

boolean  b  =  shootTed(60,  nNumberofHits  EdPS, 
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nNumberOfAttempts_EdPS ,  nNumberofHits_TedPS ) ;  //  using  half  way  between  80  and 
40 


if  (b)  nNumberofHits_TedNS++; 

} 

else  if  (Math . ceil (nNumberofHits_TedPS)  >=  10)  { 

if  (nNumberofHits_TedPS  *  Player . RATIO  > 
nNumberOfAttempts_Ted_estimated)  {  //  NOTE  inaccuracy  s.b  ">=" 

nNumberof Hits_TedNS  =  0; 
nNumberof Attempts_Ted_estimated  =  0; 
nNumberof Hits_EdNS  =  0; 
nNumberof Attempts_EdNS  =  0; 

}  else  { 

nNumberof Hits_TedNS  =  0; 
nNumberof Attempts_Ted_estimated  =  0; 

} 

} 


//==  Ed 

if  (Math .  floor (nNumberofHits_EdPS )  <  2)  { 

boolean  b  =  shootEd(50,  nNumberofHits_EdPS, 
nNumberof Attempts_EdPS ,  nNumberofHits_TedPS ) ; 

if  (b)  { 

nNumberof Hit s_EdNS++; 

} 


>=  6)  { 

nNumberofHits_EdPS , 


} 

else  if  (nNumberofHits_EdPS  <  6.0)  { 

if  (nNumberofHits^TedPS  <2  | |  nNumberof Hits 

boolean  b  =  shootEd(75, 
nNumberOfAttempts_EdPS,  nNumberof Hits_TedPS ) ; 

if  (b)  { 


nNumberof Hit s_EdNS++; 

}  else  { 


TedPS 


nNumberof Hits 
nNumberof Hits 


EdPS,  nNumberof Attempts_EdPS, 
EdNS — ; 


b  =  shootEd(25, 
nNumberof Hits_TedPS )  ; 

if  (b  &&  nNumberofHits_EdNS  >  0) 


} 

}  else  { /*lock!sMine*///  not  implemented  in  Estimator 


Math . ceil (nNumberofHits 

nNumberOfAttempts_EdPS , 
40 


nNumberOfAttempts_EdNS ) 


} 

} 

else  if  (nNumberofHits_EdPS  >=  6  && 

EdPS)  <  10)  { 

boolean  b  =  shootEd(60,  nNumberofHits^EdPS, 
nNumberofHits_TedPS ) ;  //  using  half  way  between  80  and 

if  (b)  nNumberofHits_EdNS++ ; 

} 

else  if  (Math . ceil (nNumberofHits_EdPS )  >=  10)  { 

if  (nNumberofHits_EdPS  *  Player . RATIO  > 

[  II  NOTE  inaccuracy  s.b  ">=" 

nNumberof Hits_EdNS  =  0; 
nNumberof Attempts_EdNS  =  0; 
nNumberof Hits_TedNS  =  0; 
nNumberof Attempts_Ted_estimated  =  0; 

}  else  { 

nNumberof Hits_EdNS  =  0; 
nNumberof Attempts_EdNS  =  0; 

} 

} 
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output . set  (0,  0,  nNumberofHits_EdNS ) ; 
output. set (1,  0,  nNumberOfAttempts^EdNS ) 
output . set  (2 ,  0,  nNumberofHits_TedNS) ; 

return  output; 

} 
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